Much image processing work has been applied to detecting skin pixels in a digital image. Most of this work does not attempt to produce a single estimate of the skin color a person in the image, but instead simply classifies pixels into skin and non-skin categories, using broad models of skin color that apply across many types of imaging conditions. In many cases, non-skin objects with skin-like color, such as cardboard boxes and wooden tables, are classified as skin pixels by such algorithms. Some prior methods attempt to produce a skin color estimate of a person in an image, but these typically do not account for the effects of the lighting and imaging device at the time of the image capture. Thus, the resulting skin color estimates of the same person in different images may be very different if the lighting or imaging device has been changed.
Much less work has investigated objective measurement of human skin coloration to enable its color classification. Classification of a person's skin coloration would be useful, for example, in the medical field for quantification of skin erythema, lesions, ultra-violet radiation effects, and other phenomena. In the field of computer graphics people could be rendered more accurately in video-conferencing, or their appearance could be improved or altered. In the fashion industry, automated suggestion of personal appearance products, such as clothing, makeup, and eyeglasses, that complement skin tone could be facilitated. In the field of biometrics, automatic classification of skin color could be used as an aid in recognizing a person, or used in systems in which determination of skin coloring is useful.
Prior work in the medical domain currently requires sophisticated, calibrated instrumentation and controlled lighting and is not designed to discriminate skin colors across people. In the field of computer graphics and interfaces, emphasis has been directed to representation and synthesis rather than classification of skin color, and multi-spectral data beyond what a camera normally provides is sometimes required. Other methods use a camera, colorimeter, spectrophotometer, or confocal imaging under controlled illumination to estimate skin color at a specific skin location manually selected by a human operator. Some of these methods obtain spectral reflectance values for the skin with controlled illumination provided by the device itself, thus producing a skin color representation that is independent of the ambient illumination. The disadvantage of such methods is that the capture devices used are much more expensive than a typical camera and require a trained operator.
Other prior work extracts skin color characteristics, such as bi-directional reflectance distribution functions (BRDFs), or melanin and hemoglobin content that are independent of the illuminant and the imaging device. However, that work focused on synthesis of new images of a person under different conditions, such as changed lighting, and did not attempt to extract, from their extensive measured data, a single estimate representing the skin color of the person. Instead, they obtained, in effect, a set of skin color estimates from different locations on the subject's face. The selection, combination, and reduction of these spatially-varying skin color estimates to a single estimate representing the overall skin color requires analysis of facial features and measured color statistics, and is thus non-trivial. Also, some of these methods use multiple images to measure the skin color characteristics.
Some current methods rely upon a controlled infrastructure in which the ambient lighting conditions and the camera processing parameters are carefully calibrated and controlled. Prior knowledge of the camera processing parameters and lighting conditions are necessary in these methods in order to accurately classify the skin color of a person by compensating for these effects in the picture being analyzed. One or more pictures of the subject are taken and analyzed by a human consultant who then generates, for example, a cosmetics consultation to the subject. However, due to the expense and amount of space these facilities require, they are not generally made available to most subjects. Other systems rely upon a plurality of pictures which record different locations of the subject's skin or under different lighting conditions.